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Description 



SUMMARY OF THE INVENTION 



BACKGROUND OF THE INVENTION 

Field of the Invention 

The invention relates to image coding apparatus 
and method for efficiently coding image data. 

Related Background Art 

An MPEG (Moving Picture Experts Group) system 
is widely known as a system for highly efficiently com- 
pressing and coding motion image data. The MPEG 
system is a coding system in which a discrete cosine 
transformation (DCT), an adaptive quantization, and a 
variable length coding are used as fundamental tech- 
niques and an interf rame (or interfield) predictive coding 
with motion compensation based on a frame (or field) 
correlation is adaptively combined. 

Specifically speaking, an I picture in which each 
frame is coded in a picture plane, a P picture which is 
interframe (or interfield) motion compensation predic- 
tive coded from the I picture that is preceding with re- 
spect to the time, and a B picture which is interframe (or 
interfield) motion compensation predictive coded from 
both of the preceding I picture (or P picture) and the sub- 
sequent I picture (or P picture) are arranged in accord- 
ance with a proper order, thereby accomplishing a high 
compression ratio. 

A recent image coding system has a construction 
which can be also called a set of various coding tech- 
niques. Among the various coding methods, there are 
advantages and disadvantages depending on a photo- 
graphing situation or an object. However, which coding 
method is used is proper can be discriminated for the 
first time by analyzing an image signal inputted to a com- 
pression processing circuit. 

The MPEG system is what is called a collective 
method of those composite coding methods. A camera 
integrated type recording and reproducing apparatus 
using the MPEG system has already been manufac- 
tured as an article of commerce. 

According to the MPEG system, since an inter-pic- 
ture plane correlation is used, a compression ratio is 
higher than that of a compression coding system of a 
conventional digital VCR for industrial use, a longer time 
recording can be performed with a smaller recording ca- 
pacity, and it is suitable for a portable camera apparatus. 

However, the MPEG system is not suitable to form 
a predictive coding image for a period of time of the cam- 
era operation such as zoom-in (enlargement), zoom-out 
(reduction), or the like. That is, since it is difficult to ob- 
tain a correlation between the picture planes during the 
zooming operation, a large amount of unnecessary data 
is generated due to prediction errors. Not only a coding 
efficiency deteriorates but also a picture quality deteri- 
orates. 



From a background as mentioned above, it is a con- 
cern of the invention to provide coding apparatus and 
5 method in which a photographed image which gives a 
continuous change for image data can be promptly and 
efficiently coded without deteriorating a coding efficien- 
cy and a picture quality. 

According to one preferred embodiment of the in- 
fo vention there is provided an image coding apparatus 
comprising: input means for inputting image data; image 
changing means for giving a continuous change to the 
image data; coding means for coding the image data; 
and control means for controlling coding conditions of 
15 the coding means in accordance with an output of the 
changing means. 

According to another preferred embodiment of the 
invention, there is provided an image coding method 
comprising: an image changing step of giving a contin- 
20 uous change to input image data; a coding step of cod- 
ing the image data; and a control step of controlling a 
coding condition of the coding means in accordance 
with a process in the changing step. 

Other features and advantages of the invention will 
25 become apparent from the following detailed description 
taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

30 Fig. 1 is a block diagram showing a construction of 
a digital video camera of the first embodiment ac- 
cording to the invention; 

Fig. 2 is a block diagram showing a construction of 

a digital video reproducing apparatus of the embod- 
35 iment according to the invention: 

Fig. 3 is a flowchart for the coding mode selecting 

operation of the embodiment; 

Fig. 4 is a flowchart for the coding mode selecting 

operation according to another embodiment; 
40 Fig. 5 is a block diagram showing a construction of 

a digital video camera of the second embodiment 

according to the invention; 

Figs. 6A and 6B diagrams for explaining a quanti- 
zation step control of the second embodiment; and 
45 Fig. 7 is a block diagram showing a construction of 
a digital video camera of another embodiment ac- 
cording to the invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
50 EMBODIMENTS 

The first embodiment according to the invention will 
now be described in detail hereinbelow with reference 
to the drawings. 
55 Fig. 1 is a block diagram showing a construction of 
a digital video camera of the first embodiment according 
to the invention. Reference numeral 10 denotes a pho- 
tographing lens constructed by a plurality of lens groups; 
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12 a zoom drive circuit for zooming the photographing 
lens 10; 14 an image pickup for converting an optical 
image obtained by the photographing lens 10 into an 
electric signal; 16 an A/D converter for converting an 
analog output of the image pickup 14 intoa digital signal; 
and 1 8 a camera signal processing circuit for performing 
well-known camera signal processes such as gamma 
correction, color balance correction, and ; the like to out- 
put data of the A/D converter 16. 

Reference numeral 22 denotes a switch for select- 
ing image data (a contact) of a block unit from a block 
formation circuit 20 or a prediction error (b contact) be- 
tween the actual picture plane and a predictive picture 
plane; 24 a DCT circuit for discrete cosine transforming 
an output of the switch 22 and outputting DCT coefficient 
data: 26 a quantization circuit for quantizing the DCT 
coefficient data outputted from the DCT circuit 24 by a 
designated quantization step size; 28 a variable length 
coding circuit for variable length coding an output of the 
quantization circuit 26; 30 a buffer for storing an output 
of the variable length coding circuit 28; and 32 a rate 
control circuit for controlling a quantization step of the 
quantization circuit 26 in accordance with a data amount 
in the buffer 30. 

Reference numeral 34 denotes a format circuit for 
converting data from the buffer 30 into a recording for- 
mat in a semiconductor memory 36. 

Reference numeral 38 denotes an inverse quanti- 
zation circuit for inversely quantizing the output of the 
quantization circuit 26; 40 an inverse DCT circuit for in- 
versely discrete cosine transforming an output of the in- 
verse quantization circuit 38. Reference numeral 42 de- 
notes an adder for adding a predictive value to an output 
of the inverse DCT circuit 40 and outputting a resultant 
value in an inter-picture plane predictive coding mode 
(inter-coding mode) [P picture (forward prediction) and 
B picture (two-way prediction)] or for outputting the out- 
put of the inverse DCT circuit 40 as it is in an intra-picture 
plane coding mode (intra-coding mode) (I picture). Ref- 
erence numeral 44 denotes a motion vector detection 
circuit for comparing an output of the adder 42 with im- 
age data of the present picture plane from the block for- 
mation circuit 20 and detecting a motion on a block unit 
basis of a predetermined size. Reference numeral 46 
denotes a motion compensation circuit for moving im- 
age data of a predictive picture plane in the pictu re plane 
so as to set off the motion in accordance with a motion 
vector detected by the motion vector detection circuit 44. 
An output of the motion compensation circuit 46 is sup- 
plied as a prediction value to the adder 42 through a 
subtractor 48 and a switch 50. The subtractor 48 calcu- 
lates a difference between the output of the block for- 
mation circuit 20 and the output (prediction value) of the 
motion compensation circuit 46, namely, a prediction er- 
ror and supplies to a b contact of the switch 22. 

Reference numeral 52 denotes a D/A converter for 
converting the digital output of the camera signal 
processing circuit 18 into an analog signal; and 54 an 



electronic view finder for displaying a video image ac- 
cording to an output signal of the D/A converter 52. 
Thus, the user can observe the video image during the 
photographing or during the photographing preparation. 
s Reference numeral 56 denotes a system control cir- 

cuit for controlling the whole apparatus; 58 an operation 
unit comprising operation keys for the user to input var- 
ious instructions to the system control circuit 56; and 59 
a digital interface (for example, IEEE1394). 
10 The operation of the embodiment will now be de- 
scribed. The image pickup 1 4 converts the optical image 
obtained by the photographing lens 10 into the electric 
signal. The output signal of the image pickup 14 is con- 
verted into the digital signal by the A/D converter 1 6 and 
15 is subjected to well-known camera signal processes by 
the camera signal processing circuit 18. 

The block formation circuit 20 divides the image da- 
ta outputted from the camera signal processing circuit 
18 into blocks of (8 pixels X 8 pixels). An output of the 
20 block formation circuit 20 is supplied to the a contact of 
the switch 22, subtractor 48, and motion vector detec- 
tion circuit 44. The subtractor 48 calculates a difference 
(prediction error) between the pixel data from the block 
formation circuit 20 and the prediction value outputted 
25 from the motion compensation circuit 46 and supplies 
to the b contact of the switch 22. The switch 22 is con- 
nected to the a contact in the I picture [intra-picture 
plane coding mode (intra-coding mode)] and is connect- 
ed to the b contact in the B picture [inter-picture plane 
30 coding mode (inter-coding mode)]. Therefore, in the I 
picture, the output of the block formation circuit 20 is 
supplied to the DCT circuit 24. In the P picture or the B 
picture, the output (prediction error) of the subtractor 48 
is supplied. 

35 The DCT circuit 24 discrete cosine (DCT) trans- 
forms the data from the switch 22 on a block unit basis 
and outputs DCT coefficient data to the quantization cir- 
cuit 26. The quantization circuit 26 quantizes the DCT 
coefficient data from the DCT circuit 24 by a quantization 
40 step size designated by the rate control circuit 32. The 
variable length coding circuit 28 variable length codes 
the output of the quantization circuit 26. 

The rate control circuit 32 monitors a data accumu- 
lation amount in the buffer 30 and controls the quanti- 
45 zation step size of the quantization circuit 26 so that the 
data accumulation amount does not overflow. 

The inverse quantization circuit 38 inversely quan- 
tizes the output of the quantization circuit 26 by the same 
quantization step size as that selected in the quantiza- 
50 tion circuit 26 and outputs a representative value of a 
DCT coefficient. The inverse DCT circuit 40 inversely 
discrete cosine transforms the output of the inverse 
quantization circuit 38. The switch 50 is opened in the 
intra-coding mode (I picture) and is closed in the inter- 
ns coding mode (P picture or B picture). Therefore, the 
adder 42 adds a prediction value (output of the motion 
compensation circuit 46) to the output of the inverse 
DCT circuit 40 in the inter-coding mode and outputs the 
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output of the inverse DCT circuit 40 as it is in the intra- 
coding mode. 

The motion vector detection circuit 44 compares the 
pixel data ot the present frame from the block formation 
circuit 20 with the pixel data of the prediction frame from 
the adder 42, thereby detecting a motion of the image. 
The motion compensation circuit 46 moves the pixel da- 
ta of the prediction frame from the adder 42 in the picture 
plane so as to set off the motion in accordance with the 
motion vector detected by the motion vector detection 
circuit 42. The output of the motion compensation circuit 
46 is supplied as a prediction value to the adder 42 
through the subtractor 48 and switch 50. 

The output of the variable length coding circuit 28 
is rate adjusted by the buffer 30 and is supplied to the 
format circuit 34. The format circuit 34 formats the data 
from the buffer 30 into a recording format of the semi- 
conductor memory 36 and supplies the resultant data to 
the semiconductor memory 36. Specifically speaking, 
the format circuit 34 adds various headers such as slice 
header, sequence header, and the like in the data struc- 
ture of MPEG1 to the coding data and forms MPEG im- 
age data corresponding to the MPEG1 system and 
writes into the semiconductor memory 36. 

The output of the camera signal processing circuit 
18 is converted into the analog signal by the D/A con- 
verter 52 and is supplied to the electronic viewfinder 54. 
Thus, the user can monitor the image during the photo- 
graphing preparation or the image to be recorded. 

The coding data outputted from the buffer 30 can 
be outputted to an external apparatus via the digital in- 
terface 59. 

Fig. 2 shows a schematic constructional block dia- 
gram of the reproducing system for reproducing the im- 
age recorded in the semiconductor memory 36 as men- 
tioned above. Reference numeral 60 denotes an input 
terminal of the coding data read out from the semicon- 
ductor memory 36; 62 an input buffer; 64 a variable 
length decoding circuit; 66 an inverse quantization cir- 
cuit: 68 an inverse DCT circuit; 70 an adder for adding 
the prediction value to an output of the inverse DCT cir- 
cuit 68; 72 a switch for selecting the output (a contact) 
of the inverse DCT circuit 68 in case of the I picture and 
selecting an output (b contact) of the adder 70 in case 
of the P picture or the B picture on the basis of identifi- 
cation information of the coding system; 74 a motion 
compensation circuit for outputting a prediction value 
which is motion compensated from an output of the 
switch 72; 76 an output buffer for storing the output of 
the switch 72; 78 a D/A converter for converting an out- 
put of the output buffer 76 into an analog signal; 80 a 
monitor for displaying a video image from an output of 
the D/A converter 78; 82 a digital interface (for example, 
I EEE 1 394) for outputting digital output data of the output 
buffer 76 to the outside; and 84 a digital output terminal. 

The coding data inputted to the input terminal 60 is 
supplied to the variable length decoding circuit 64 
through the input buffer 62 and is variable length decod- 



ed. An output of the variable length decoding circuit 64 
is inversely quantized by the inverse quantization circuit 
66 and is inversely DCT transformed by the inverse DCT 
circuit 68. 

s An output of the inverse DCT circuit 68 of the I pic- 
ture is transferred as it is to the output buffer 76 and 
motion compensation circuit 74 through the switch 72. 
An output of the inverse DCT circuit 68 of the P picture 
or the B picture is a differential signal between the 
10 picked-up frame and the prediction frame. After the pre- 
diction value was added to the output of the inverse DCT 
circuit 68 by the adder 70, a resultant value is sent to 
the output buffer 76 and motion compensation circuit 74 
through the switch 72. The motion compensation circuit 
15 74 calculates the prediction value to be added by the 
adder 70 from the data (reconstructed image data) from 
the switch 72 and supplies to the adder 70. 

The output buffer 76 converts the image data ac- 
cording to the block order for compression coding in ac- 
20 cordance with the raster order and generates the con- 
verted data. The output of the output buffer 76 is sup- 
plied to the D/A converter 78 and digital interface 82. 
The D/A converter 78 converts the output of the output 
buffer 76 into the analog signal and supplies to the mon- 
25 itor 80 such as a liquid crystal display apparatus or the 
like. Thus, a reproduction image is displayed. The digital 
interface 82 outputs the data from the buffer 76 to the 
digital output terminal 84 in a predetermined format. 
In such a camera integrated type recording and re- 
30 producing apparatus, when an object image is optically 
zoomed in/out (enlarged/reduced) ; the operator oper- 
ates a zoom lever of the operation unit 58 while moni- 
toring the image. In accordance with the zooming oper- 
ation, the system control circuit 56 moves a lens for 
35 zooming of the photographing lens 1 0 in the optical axis 
direction through the zoom drive circuit 12, thereby 
changing to a desired angle of view. The system control 
circuit 56 also forcedly connects the switch 22 to the 
contact for a zooming period of time. Thus, all of the 
40 images become the I pictures for the zooming period of 
time and an increase in unnecessary data amount which 
is caused because it becomes difficult to obtain a cor- 
relation between the picture planes in the continuous 
enlarging or reducing step of the angle of view during 
45 the zooming operation can be prevented. Since the im- 
ages which are recorded are only the I pictures, the pic- 
ture quality is also improved. 

A coding mode control during the zooming opera- 
tion will now be described in detail with reference to Fig. 
50 3. The camera integrated type recording and reproduc- 
ing apparatus of the embodiment is set into a photo- 
graphing mode and the photographing operation is 
started. First, a parameter which is used for coding is 
initialized (step S1). At this stage, any one of the I pic- 
55 ture, P picture, and B picture can be used. In accordance 
with an instruction to start the recording by the user, the 
recording of the photographed image, namely, a coding 
process of the photographed image is started (S2). 
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The presence or absence of the zooming operation 
is always monitored (S3). When the zooming operation 
(zoom-in or zoom-out) is executed, the apparatus is 
fixed to a coding mode of only the I picture (S4) and the 
photographed image is coded (S5). When the zooming 
operation is not performed, the photographed image is 
coded in a coding mode using the I picture, P picture, 
and B picture in the ordinary manner (S6). 

The processes after step S1 are repeated (S6) until 
the operation to finish the photographing (recording) is 
performed. The photographing is finished by the photo- 
graphing finishing operation. 

In the embodiment, as mentioned above, since the 
process is limited to the coding process of only the I pic- 
ture for the zooming operation, an extreme increase in 
coding data amount during the zooming operation can 
be prevented, so that the deterioration of the image can 
be suppressed. 

In association with the high magnification of the 
photographing optical system, the embodiment is suit- 
able, particularly, for a case where the zoom can be con- 
trolled at an extremely high speed. The invention can 
be obviously applied to an electronic zoom besides the 
optical zoom. 

Although the embodiment using the semiconductor 
memory as a recording medium has been described, a 
magnetic tape, a magnetic disk, or a magnetooptic disk 
can be also used as a recording medium. 

The image compressing system is not limited to the 
MPEG1 system but can also use an MPEG2 system, or 
further, another coding system using both of the inter- 
picture plane coding and the intra-picture plane coding. 

In recent years, since a zoom magnification is in- 
creasing and a request for a moving time between the 
wide angle end and the telescope end is almost un- 
changed, a zooming speed is eventually raised. On the 
other hand, since a trimming of a delicate angle of view 
is also required, a low speed driving has to be also en- 
abled. In the above embodiment, since only the I picture 
is used during the zooming operation irrespective of the 
zooming speed, when the zooming speed is low, a cod- 
ing efficiency deteriorates. To solve such a problem, it 
is sufficient to select the coding mode in accordance 
with the zooming speed. Fig. 4 shows a flowchart for the 
coding mode selection according to a modification. 

First, the parameter which is used for coding is ini- 
tialized (S11 ). At this stage, any one of the I picture, P 
picture, and B picture can be used. In accordance with 
the instruction to start the recording by the user, the re- 
cording of the photographed image, namely, a coding 
process of the photographed image is started (S12). 

The presence or absence of the zooming operation 
is always monitored (S1 3). When the zooming operation 
(zoom-in or zoom-out) is executed, the zooming speed 
is examined (S14). When the zooming speed is high, 
the apparatus is fixed to the coding mode of only the I 
picture (S15) and the photographed image is coded 
(S17). When the zooming speed is middle, the appara- 



tus is set into the coding mode using the I picture and 
the P picture (S16). The photographed image is coded 
(S17). When the zooming speed is low or the zooming 
operation is not performed (S13, S14), the photo- 
5 graphed image is coded in the coding mode using the I 
picture, P picture, and B picture in the ordinary manner 
(S17). 

The processes after step S11 are repeated (S18) 
until the operation to finish the photographing (record- 
ing) is performed. The photographing is finished by the 
photographing finishing operation. 

Hitherto, the presence or absence of the correlation 
between the picture planes has been discriminated by 
the detection result of the motion vector between the pic- 
ture planes. However, in a case where the whole picture 
plane changes while changing the angle of view like a 
zoom photographing, it is difficult to properly discrimi- 
nate the correlation between the picture planes by the 
simple block matching. Like the embodiment, by con- 
trolling an image compressing condition in accordance 
with photographing condition information from the cam- 
era system, an image compression can be efficiently 
performed at a high picture quality. 

The second embodiment according to the invention 
will now be described hereinbelow with reference to the 
drawings. 

Fig. 5 is a block diagram showing a construction of 
a digital video camera of the second embodiment ac- 
cording to the invention. In Fig. 5, the same portions as 
those in Fig. 1 are designated by the same reference 
numerals and their descriptions are omitted. 

In the second embodiment, in addition to the coding 
mode control by the zooming operation in the first em- 
bodiment, a quantization step size is further controlled. 
The embodiment is made by paying attention to an idea 
such that when the zooming operation is being per- 
formed, the photographer pays an attention to an object 
existing at a far end of the zoom and he ought to obvi- 
ously desire that a person who observes the video im- 
age also pays an attention to such a target portion. That 
is, since the center of the picture plane is a focal point 
of the zoom, a quantization step size of the image data 
near the center of the picture plane is set to be smaller 
than that of the other portion, thereby allocating a larger 
code amount to such image data. In other words, a pic- 
ture quality of the image in the center portion. This also 
provides an effect of preventing a raid increase in code 
amount due to the change in coding mode. 

Specifically speaking, a system control circuit 56' 
supplies zoom operation information (also including in- 
formation of the zooming speed and the like) showing 
the contents of the zooming operation to a rate control 
circuit 32'. The rate control circuit 32' controls the quan- 
tization step size which is used for a quantizing process 
in consideration of the zooming operation information. 

Figs. 6A and 6B are diagrams for explaining the re- 
lation among the quantization step sizes at the time of 
the zooming operation. 
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In Figs. 6A and 6B, one picture plane is divided into 
three areas concentrically from the center of the picture 
plane and the quantization step size is reduced as the 
position approaches the center of the picture plane. In 
the embodiment, although one picture plane has been 
divided into three areas, it can be also further finely di- 
vided into the areas. 

Fig. 6A shows a case of the middle zooming speed. 
Fig. 6B shows a case of the high zooming speed. When 
the zooming speed is low, the quantization step is exe- 
cuted by an ordinary rate control process. 

Although the zooming process of the image has 
been optically performed in the first and second embod- 
iments, the invention can be also applied to a case of 
electrically performing the zooming process (electronic 
zoom) instead of the optical method. Such a construc- 
tional example is shown in Fig. 7. It is sufficient that the 
system control circuit 56' controls an electronic zoom 
circuit 100 in accordance with the operation of the zoom 
lever of the operation unit 58. 

In other words, the foregoing description of embod- 
iments has been given for illustrative purposes only and 
not to be construed as imposing any limitation in every 
respect. 

The scope of the invention is, therefore, to be de- 
termined solely by the following claims and are not lim- 
ited by the text of the specifications and alterations 
made within a scope equivalent to the scope of the 
claims fall within the true spirit and scope of the inven- 
tion. 



Claims 

1. An image coding apparatus comprising: 

a) input means for inputting image data; 

b) image changing means for giving a continu- 
ous change to said image data; 

c) coding means for coding said image data; 
and 

d) control means for controlling a coding con- 
dition of said coding means in accordance with 
an output of said changing means. 

2. An apparatus according to claim 1 , wherein said in- 
put means includes image pickup means for con- 
verting an optical image into an electric signal and 
outputting as image data. 

3. An apparatus according to claim 2, wherein said im- 
age changing means is means for controlling a 
zoom lens equipped for said image pickup means. 

4. An apparatus according to claim 1 , wherein said im- 
age changing means is means for electrically zoom 
processing said image data. 



5. An apparatus according to claim 1, wherein said 
coding means has an intra-coding mode and an in- 
ter-coding mode and said coding condition is said 
coding mode. 

5 

6. An apparatus according to claim 5, wherein said 
coding means is based on an MPEG (Moving Pic- 
ture Experts Group) system. 

10 7. An apparatus according to claim 5, wherein said 
control means controls said coding condition in ac- 
cordance with a changing speed of said image 
changing means. 

15 8. An apparatus according to claim 5, wherein said im- 
age changing means is means for performing a 
zooming process to an image. 

9. An apparatus according to claim 1, wherein said 
20 coding means has quantizing means for quantizing 

said image data and said coding condition is a 
quantization step of said quantizing means. 

10. An apparatus according to claim 9, wherein said 
25 control means controls said coding condition in ac- 
cordance with a changing speed of said image 
changing means. 

11. An apparatus according to claim 9, wherein said im- 
30 age changing means is means for performing a 

zooming process to an image. 

12. An apparatus according to claim 11, wherein said 
control means controls the quantization step so as 

35 to be finer in a center direction of said picture plane 
in accordance with an output of said image chang- 
ing means. 

13. An apparatus according to claim 1 , further compris- 
40 ing recording means for recording the image data 

coded by said coding means to a recording medi- 
um. 

14. An apparatus according to claim 1 , further compris- 
45 ing a digital interface for transmitting the image data 

coded by said coding means to an external appa- 
ratus. 

15. A digital video camera having an image coding ap- 
50 paratus according to claim 1 . 

16. An image coding method comprising: 

an image changing step of giving a continuous 
55 change to input image data; 

a coding step of coding said image data; and 
a control step of controlling a coding condition 
of said coding means in accordance with a 
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process in said changing step. 
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